feat(ci): Add comprehensive performance tracing with metrics analysis#13
Merged
neil-pozetroninc merged 6 commits intomainfrom Oct 25, 2025
Merged
feat(ci): Add comprehensive performance tracing with metrics analysis#13neil-pozetroninc merged 6 commits intomainfrom
neil-pozetroninc merged 6 commits intomainfrom
Conversation
- Add dotnet tools directory to PATH for subsequent steps - Switch from --providers to --profile cpu-sampling for actual CPU profiling - This generates flamegraphs with real stack traces instead of empty traces Previous approach only captured runtime events (GC, JIT) without CPU samples. The cpu-sampling profile includes Microsoft-DotNETCore-SampleProfiler which captures stack traces at 1ms intervals, enabling proper performance analysis. Tested locally with act - generates 22KB trace with actual profiling data. Co-Authored-By: Claude <noreply@anthropic.com>
8655a1e to
84d833d
Compare
…rical tracking - Add webhook workload simulation to trace real request processing - Create analyze-trace.py script to extract performance metrics - Implement baseline comparison against main branch via Actions cache - Add historical performance tracking in perf-history/ directory - Display performance reports in GitHub Actions summary - Upload metrics artifacts with 30-day retention This provides actionable performance insights and trend tracking over time, enabling early detection of performance regressions.
The analyze-trace.py script was only parsing the 'sampled' profile format (samples/weights arrays), but dotnet-trace produces 'evented' format (open/close events with timestamps). This caused the script to extract 0 samples from valid trace files. Changes: - Add support for evented profile type with O/C event parsing - Track stack state and calculate frame durations from events - Fix deprecated datetime.utcnow() usage - Maintain backward compatibility with sampled format The evented format is parsed by tracking open/close events to reconstruct the call stack and measure time each frame spent active. This produces accurate hotspot metrics from dotnet-trace output.
Add a step to post performance metrics as a PR comment, making them immediately visible on the PR page without requiring users to click through to job details. The comment includes: - Full performance summary with metrics - Link to detailed trace in Actions - Link to downloadable flamegraph artifact Only runs on pull requests to avoid errors on direct pushes.
📊 Performance Trace ReportPerformance Trace AnalysisSummary
Top 10 Hotspot Methods
|
Significantly improved readability and usability of performance trace reports: **Readability Improvements:** - Human-readable time formats (41.6 min vs 2498937ms) - Visual progress bars for percentages using block characters - Clear verdict at top (✅/⚠️ /🔴) based on baseline comparison - Collapsible sections to reduce information density **Balanced View:** - Show verdict + key metrics prominently - Filter application hotspots from framework noise - Categorize methods as app/framework/noise with emoji badges - Full technical details available in collapsible sections **New Features:** - format_duration(): Convert ms to readable format - format_percentage_bar(): Create visual █░ progress bars - categorize_method(): Detect ProbotSharp app code vs framework - Improved baseline comparison with trend indicators (↑↓→) - Contextual help: "Changes within normal variance" guidance **User Experience:** - Before: Dense technical table, unclear if performance is good/bad - After: Clean verdict, visual bars, filtered hotspots, actionable insights Tested with real trace files - output is much more scannable and actionable.
📊 Performance Trace Report📊 Performance SummaryVerdict: ✅ No significant performance impact detected Key Metrics
🔥 Application HotspotsNo significant application hotspots detected. Most time spent in framework/wait operations. 📋 Full Method Details (all categories)
🔍 Technical Details
|
…nalysis Fixed critical bug where script summed inclusive times instead of calculating exclusive times, causing massive overcounting (30s trace showed as 41 minutes). Changes: - Implemented stack-based exclusive time calculation for evented format - Track both inclusive (total) and exclusive (self) time per frame - Calculate wall-clock duration as max(profile durations), not sum - Total CPU time = sum of exclusive times across all frames - Update all output fields to use new metric names Metrics: - wall_clock_ms: Actual elapsed time (endValue - startValue) - total_cpu_time_ms: Sum of all exclusive times - Methods show both inclusive_ms and exclusive_ms - Percentages calculated against wall_clock, not summed times Verification: Before: 2,498,937 ms (41 minutes) - WRONG After: 30,016 ms (30 seconds) - CORRECT The fix follows industry best practices (Brendan Gregg, Speedscope) for calculating exclusive time: exclusive = inclusive - children_time.
📊 Performance Trace Report📊 Performance SummaryVerdict: ✅ No significant performance impact detected Key Metrics
🔥 Application HotspotsNo significant application hotspots detected. Most time spent in framework/wait operations. 📚 Framework Methods (click to expand)
📋 Full Method Details (all categories)
🔍 Technical Details
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements comprehensive performance monitoring and analysis for the CI pipeline, building on the initial CPU profiling foundation.
Key Features
Real Webhook Workload Simulation
Automated Metrics Extraction
scripts/analyze-trace.pyparses speedscope tracesBaseline Comparison
Historical Performance Tracking
perf-history/directoryGitHub Actions Summary Integration
Changes
.github/workflows/dotnet.yml: Added webhook simulation, analysis, and reporting stepsscripts/analyze-trace.py: New metrics extraction and analysis toolTest Plan
gh act- all steps execute successfullyBenefits